277 research outputs found
From Rank Estimation to Rank Approximation: Rank Residual Constraint for Image Restoration
In this paper, we propose a novel approach to the rank minimization problem,
termed rank residual constraint (RRC) model. Different from existing low-rank
based approaches, such as the well-known nuclear norm minimization (NNM) and
the weighted nuclear norm minimization (WNNM), which estimate the underlying
low-rank matrix directly from the corrupted observations, we progressively
approximate the underlying low-rank matrix via minimizing the rank residual.
Through integrating the image nonlocal self-similarity (NSS) prior with the
proposed RRC model, we apply it to image restoration tasks, including image
denoising and image compression artifacts reduction. Towards this end, we first
obtain a good reference of the original image groups by using the image NSS
prior, and then the rank residual of the image groups between this reference
and the degraded image is minimized to achieve a better estimate to the desired
image. In this manner, both the reference and the estimated image are updated
gradually and jointly in each iteration. Based on the group-based sparse
representation model, we further provide a theoretical analysis on the
feasibility of the proposed RRC model. Experimental results demonstrate that
the proposed RRC model outperforms many state-of-the-art schemes in both the
objective and perceptual quality
Generalizable Synthetic Image Detection via Language-guided Contrastive Learning
The heightened realism of AI-generated images can be attributed to the rapid
development of synthetic models, including generative adversarial networks
(GANs) and diffusion models (DMs). The malevolent use of synthetic images, such
as the dissemination of fake news or the creation of fake profiles, however,
raises significant concerns regarding the authenticity of images. Though many
forensic algorithms have been developed for detecting synthetic images, their
performance, especially the generalization capability, is still far from being
adequate to cope with the increasing number of synthetic models. In this work,
we propose a simple yet very effective synthetic image detection method via a
language-guided contrastive learning and a new formulation of the detection
problem. We first augment the training images with carefully-designed textual
labels, enabling us to use a joint image-text contrastive learning for the
forensic feature extraction. In addition, we formulate the synthetic image
detection as an identification problem, which is vastly different from the
traditional classification-based approaches. It is shown that our proposed
LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved
generalizability to unseen image generation models and delivers promising
performance that far exceeds state-of-the-art competitors by +22.66% accuracy
and +15.24% AUC. The code is available at https://github.com/HighwayWu/LASTED
Rethinking Image Forgery Detection via Contrastive Learning and Unsupervised Clustering
Image forgery detection aims to detect and locate forged regions in an image.
Most existing forgery detection algorithms formulate classification problems to
classify pixels into forged or pristine. However, the definition of forged and
pristine pixels is only relative within one single image, e.g., a forged region
in image A is actually a pristine one in its source image B (splicing forgery).
Such a relative definition has been severely overlooked by existing methods,
which unnecessarily mix forged (pristine) regions across different images into
the same category. To resolve this dilemma, we propose the FOrensic ContrAstive
cLustering (FOCAL) method, a novel, simple yet very effective paradigm based on
contrastive learning and unsupervised clustering for the image forgery
detection. Specifically, FOCAL 1) utilizes pixel-level contrastive learning to
supervise the high-level forensic feature extraction in an image-by-image
manner, explicitly reflecting the above relative definition; 2) employs an
on-the-fly unsupervised clustering algorithm (instead of a trained one) to
cluster the learned features into forged/pristine categories, further
suppressing the cross-image influence from training data; and 3) allows to
further boost the detection performance via simple feature-level concatenation
without the need of retraining. Extensive experimental results over six public
testing datasets demonstrate that our proposed FOCAL significantly outperforms
the state-of-the-art competing algorithms by big margins: +24.3% on Coverage,
+18.6% on Columbia, +17.5% on FF++, +14.2% on MISD, +13.5% on CASIA and +10.3%
on NIST in terms of IoU. The paradigm of FOCAL could bring fresh insights and
serve as a novel benchmark for the image forgery detection task. The code is
available at https://github.com/HighwayWu/FOCAL
Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples
With the increasing prevalence of cloud computing platforms, ensuring data
privacy during the cloud-based image related services such as classification
has become crucial. In this study, we propose a novel privacypreserving image
classification scheme that enables the direct application of classifiers
trained in the plaintext domain to classify encrypted images, without the need
of retraining a dedicated classifier. Moreover, encrypted images can be
decrypted back into their original form with high fidelity (recoverable) using
a secret key. Specifically, our proposed scheme involves utilizing a feature
extractor and an encoder to mask the plaintext image through a newly designed
Noise-like Adversarial Example (NAE). Such an NAE not only introduces a
noise-like visual appearance to the encrypted image but also compels the target
classifier to predict the ciphertext as the same label as the original
plaintext image. At the decoding phase, we adopt a Symmetric Residual Learning
(SRL) framework for restoring the plaintext image with minimal degradation.
Extensive experiments demonstrate that 1) the classification accuracy of the
classifier trained in the plaintext domain remains the same in both the
ciphertext and plaintext domains; 2) the encrypted images can be recovered into
their original form with an average PSNR of up to 51+ dB for the SVHN dataset
and 48+ dB for the VGGFace2 dataset; 3) our system exhibits satisfactory
generalization capability on the encryption, decryption and classification
tasks across datasets that are different from the training one; and 4) a
high-level of security is achieved against three potential threat models. The
code is available at https://github.com/csjunjun/RIC.git.Comment: 23 pages, 9 figure
Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
Deep neural networks (DNNs) have been shown to be vulnerable against
adversarial examples (AEs), which are maliciously designed to cause dramatic
model output errors. In this work, we reveal that normal examples (NEs) are
insensitive to the fluctuations occurring at the highly-curved region of the
decision boundary, while AEs typically designed over one single domain (mostly
spatial domain) exhibit exorbitant sensitivity on such fluctuations. This
phenomenon motivates us to design another classifier (called dual classifier)
with transformed decision boundary, which can be collaboratively used with the
original classifier (called primal classifier) to detect AEs, by virtue of the
sensitivity inconsistency. When comparing with the state-of-the-art algorithms
based on Local Intrinsic Dimensionality (LID), Mahalanobis Distance (MD), and
Feature Squeezing (FS), our proposed Sensitivity Inconsistency Detector (SID)
achieves improved AE detection performance and superior generalization
capabilities, especially in the challenging cases where the adversarial
perturbation levels are small. Intensive experimental results on ResNet and VGG
validate the superiority of the proposed SID
- …